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BLOCK-BASED IMAGE SEGMENTATION METHOD AND SYSTEM 

5 FIELD OF THE INVENTION 

This invention relates, generally, to an image 
segmentation method and system, and more particularly, 
to a method and system for merging segments of a number 
10 of image blocks or sub-images into segments of the 

entire image. 

BACKGROUND OF THE INVENTION 

15 Image segmentation or partitioning is often used 

for image analysis, processing and pattern recognition 
of video information or still pictures. In general, 
segmentation can be defined as the decomposition of an 
image into segments or regions that are homogeneous in 

20 terms of a set of specified image features. The regions 

are semantically meaningful in the context of the 
environment in which the segmentation output is to be 
used. The selection of the feature set is usually 
application dependent so that the regions defined by 

2 5 the attributes are meaningful in that application. For 

images of most natural sceneries, luminance, 
chrominance and/or texture are often used as the 
features . 

Furthermore, segmentation attempts to recover the 
30 scenery from an image. It is usually a non-linear 
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optimisation process over the feature space that is 
used to define the homogeneity of the segments of the 
image. For instance, colour image segmentation aims to 
minimise the difference in colour within each segment 
5 and maximise the difference in colour between segments. 

A typical segmentation algorithm is recursive and 
begins by checking the homogeneity between a pixel and 
its neighbours. Pixels that have homogeneous features 
and are spatially connected together are grouped into 

10 the same segments or regions. It is well documented and 

reported that the global information representing an 
image should be used in order to achieve meaningful 
segments. This practice often leads to large memory 
requirements to store the entire image and 

15 computational overheads to achieve a desirable 

segmentation . 

The efficiency of an algorithm is typically given 
by a measure of its complexity (execution time) and 
memory requirement. One typical segmentation algorithm 
20 is based on a shortest spanning tree (SST) technique 

and its efficiency measurements are: 

complexity in the order of ^ denoted as 

0(iVO; and 

memory requirement in the order of kN , denoted 
25 as 0{kN) 

where 

N is the size of the image in pixels; 
p is a constant of a value greater than or 
equal to 2 ; and 
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k is a constant of a value greater than 1. 
Such an algorithm may take an unacceptable amount 
of time to segment an image or video frame on a device 
with limited auxiliary memory and processing power. 
5 Furthermore, for a video input comprising a 

sequence of image frames, the requirement for an entire 
video frame to be buffered before the segmentation can 
be initiated implies that a large latency will be 
incurred by a segmentation process. 

10 In this specification, including the claims, the 

terms '"comprises", ''comprising^' or similar terms are 
intended to mean a non-exclusive inclusion, such that a 
method or apparatus that comprises a list of elements 
does not include those elements solely, but may well 

15 include other elements not listed. 

BRIEF SUiyiMARY OF THE INVENTION 

According to one aspect of the invention, there is 
20 provided a method of image segmentation involving the 

following steps. The method first partitions at least 
part of an input image into a plurality of partitioned 
units. The method next determines segments for each of 
the plurality of partitioned units based on at least 
25 one pixel attribute of the input image • Subsequently, 

the method selectively combines the segments of the 
partitioned units to provide a segmented version of the 
input image . 
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Preferably, the step of selectively combining 
should be effected by a shortest spanning tree 
technique . 

Suitably, the step of selectively combining may 
5 include the following steps. In performing the step of 

selectively combining, the method represents the 
segments for each of the plurality of partitioned units 
as nodes of a tree connected via links. Each of the 
links has a weight based on the at least one pixel 
10 attribute. The method next finds a least weight link 

and combines two nodes connected by the least weight 
link to form a merged node. The method then connects 
the merged node to nodes adjacent the two nodes via new 
weighted links. The method repeats the steps of 
15 finding, combining and connecting until a predetermined 

number of nodes representing the segmented version of 
the input image remain in the tree. 

Suitably, the step of partitioning may further 
include a step of generating connectivity information 
20 associated with the partitioned units. 

Suitably, the nodes of the tree may be connected 
using the connectivity information. 

Preferably, the partitioned units should include 
square blocks. 

25 Preferably, the step of determining segments 

should be effected by a shortest spanning tree 
technique . 

According to another aspect of the invention, 
there is provided a system for image segmentation 
3 0 having an image partition module, a block segmentation 
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module coupled to the image partition module and a 
segment combination module coupled to the block 
segmentation module. In use the image partition module 
partitions at least part of an input image into a 
5 plurality of partitioned units, the block segmentation 

module determines segments for each of the plurality of 
partitioned units based on at least one pixel attribute 
of the input image and the segment combination module 
selectively combines the segments of the partitioned 
10 units to provide a segmented version of the input 

image . 

Preferably, the system should further include a 
feature extraction module coupled to the block 
segmentation module. In use the feature extraction 
15 module determines the at least one pixel attribute of 

the input image . 

BRIEF DESCRIPTION OF THE DRAWINGS 

20 In order that the invention may be readily 

understood and put into practical effect reference will 
now be made to a preferred embodiment as illustrated in 
the following drawings in which: 

FIG. 1 shows a schematic block diagram of an 
25 image segmentation system according to the present 

invention; 

FIG. 2 shows an example of partitioning an 
image into image blocks ; 

FIG. 3 shows a flow chart of a method for 
3 0 segmenting an image block of FIG. 2 using a 
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shortest spanning tree (SST) technique on a 
weighted network; 

FIG. 4 shows a flow chart of a method for 
combining segments of the image blocks into 
5 segments of the entire image of FIG. 2 using a SST 

technique; 

FIG. 5 shows an example of an image 
consisting of 3 6 pixels arranged as 6 rows by 6 
columns divided into four image blocks; 
10 FIG. 6 shows how the method in FIG. 3 

segments an upper left image block of the image in 
FIG. 5; and 

FIG. 7 shows how the method in FIG. 4 
combines the segments of the four image blocks of 
15 FIG. 5 to obtain the segments of the entire image 

of FIG. 5. 
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DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT OF THE 

INVENTION 



In FIG.l there is illustrated a block based image 
segmentation system 1 including an input terminal T 
coupled to an image partition module 2 . The image 

partition module 2 has an output coupled to a block 

25 segmentation module 4, a segment combination module 5 

and a feature extraction module 3, An output of the 
feature extraction module 3 is coupled to both the 
image partition module 2 and the block segmentation 
module 4. The block segmentation module 4 has an output 

3 0 coupled to the segment combination module 5 that has an 
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output 8 . The segment combination module 5 includes a 
processing unit 7 and an intermediate data storage unit 
6 coupled to each other by a bi-directional data bus. 
In use, the image partition module 2 divides or 
5 partitions an image received at the input terminal T 

into smaller partitioned units (PUs) , so that the block 
segmentation module 4 can segment each block 
independently using limited memory and processing 
power . 

10 In FIG. 2, there is shown an example of 

partitioning an image 9 into the PUs, called image 
blocks 10, by the image partition module 2. Although 
square image blocks 10 have been illustrated, 
partitioning may be more complex and can result in PUs, 

15 each of which may be of any shape and size. The image 

partition module 2 also sends connectivity information 
of the image blocks 10 to the segment combination 
module 5. 

The image blocks 10 generated by the image 
20 partition module 2 are then passed to the feature 

extraction module 3 where features used to define the 
characteristic of image segments are obtained from 
pixel attributes within each image block 10. The 
attributes of pixel i are usually represented as a 
25 vector, f,=(fa,f,2^'"ftp)r where f^j is the j*''' component. 

The attributes include luminance, chrominance and 
texture. Colour segmentation uses pixel luminance and 
chrominance and textural segmentation uses pixel 
texture for segmentation. Any existing feature 
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extraction technique can be used to implement the 
feature extraction module 3 . 

Referring again to FIG.l, the block segmentation 
module 4 decomposes each image block 10 into segments 
5 or regions using the feature or features extracted by 

the feature extraction module 3. Alternatively, the 
feature or features may be predetermined; in which 
case, the feature extraction module 3 is optional. Any 
knovm segmentation technique, such as region splitting 

10 and growing, pixel classification, edge detection and 

shortest spaning tree, can be used or modified to 
implement the block-segmentation module 4. An example 
of how a block is decomposed into segments will be 
described later. 

15 In FIG. 3 there is illustrated a method of 

segmenting the image 9 by using a SST technique on a 
weighted network. The method is effected by the block 
segmentation module 4. The image 9 is mapped into the 
weighted network in Step 13 . An example of such a 

20 network is shown in FIG. 6. Each node in the network 

represents a pixel and an edge or a link between two 
nodes represents two corresponding pixels that are 
spatially connected. For an image represented as an 
array of pixels on a grid in 2D space, two pixels are 

25 defined to be spatially connected if the pixels are 

orthogonally adjacent to each other in the case of a 4- 
connectivity network and orthogonally or diagonally 
adjacent to each other in the case of an 8-connectivi ty 
network. The following example shows pixels X spatially 

3 0 connected to a pixel A in a 4-connectivity network. 



X 

X A X 
X 

In contrast, the pixels X spatially connected to a 
pixel A in an 8-connectivity network would be as 
follows : 

XXX 

X A X 

XXX 

The weight of the link connecting a pixel i and a 
pixel J is a function of their attribute vectors, 
and f . Euclidean distance is often used as the weight 
measurement. The network is stored in a priority queue. 

After the network is mapped and saved in the 
priority queue in Step 13, any existing algorithm, such 
as the Kruskal's algorithm, may be applied to find its 
minimum or SST in Step 14 and the SST is passed to Step 
15. Information on the Kruskal's algorithm can be found 
in most books on algorithms, such as " Algorithmics : 
Theory and Practice," Brassard and Bratley, Prentice 
Hall, 1988. In Step 15, the SST is cut into R sub-trees 
at the R-1 most costly links if it is predetermined 
that R segments or regions of the image block 10 are 
required. Alternatively, the SST can be cut at the 
links whose weights are above a predetermined 
threshold. The R sub-trees are mapped back onto R 
segments in image space in Step 16 and the R segments 
are output to the intermediate storage unit 6 in the 
segment combination module 5. This SST based block 
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segmentation method will be illustrated in detail 
later . 

Referring again to FIG.l, the segment combination 
module 5 includes two units: the intermediate data 
5 storage 6 and the processing unit 7. An input of the 

intermediate data storage 6 is coupled to an output of 
the block segmentation module for storing segments of 
all image blocks 10 generated by the block segmentation 
module 4. After all image blocks 10 have been segmented 

10 by the block segmentation module 4, the processing unit 

7 starts to selectively combine segments of all image 
blocks 10 into segments of the entire image according 
to certain criteria to produce a segmented version of 
the image. In general, combining of segments can start 

15 as soon as two blocks have been segmented. The 

processing unit 7 combines all block segments to form 
the segments of the entire image using the connectivity 
information and compatible optimisation criteria used 
in performing block segmentation. The quality of 

2 0 segmentation by combining block segments should 

preferably be comparable with segments obtained by 
applying the same segmentation algorithm to the entire 
image. The processing unit 7 utilises a global 
information representing the image 9 to transform the 
25 selectively combined block segments into the image 

segments . 

The operation of the segment combination module 5 
is illustrated in FIG. 4. which shows a method 18 for 
combining segments. At a mapping step 19, all block 

3 0 segments are mapped into a weighted network according 
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to connectivity information of segments within a block 
and across blocks in the processing unit 7. In the 
network, a node represents a block segment and a link 
represents the connectivity between two block segments. 
5 The weight associated with a link is a measurement of 

the degree of homogeneity between the two block 
segments connected by the link. 

The connectivity information of segments within a 
block is passed from the block segmentation module 4 

10 either explicitly or implicitly. The connectivity 

information of segments across blocks is passed from 
the image partition module 2 either in an explicit 
format for blocks of irregular shapes or in an implicit 
format for regular blocks, such as rectangular or 

15 square blocks. 

The weight of a link between segments Ri and Rj 
with attribute vectors, and , is calculated as a 

Euclidean distance between and fj, d{f-,f^), multiplied 

by a factor depending on the sizes of the segments, Ri 
20 and J^^. 

where Si and Sj are the sizes (in pixels) of segments Ri 
and Rj respectively. 

The processing unit 7 finds an SST of the network 
25 stored in a priority queue in a recursive manner 

comprising Steps 20, 21, 22, 23, and 24. Step 20 
involves a sorting process for selecting a link with a 
least weight. The selected link is saved and the two 
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nodes connected by the link are merged into a new node 
y in Step 21. In this regard, let V. and Vj be the two 

nodes connected by the selected link and f- and fj be 

their feature vectors respectively. The newly merged 
5 node shall have an attribute vector 

where and Sj are the sizes (in pixels) of the nodes 

and respectively. 

In Step 22, the newly merged node V is used to 

10 replace nodes V. and . All unprocessed links that 

previously connect to nodes V. and are now connected 

to the new node V, The weights of these unprocessed 
links are recalculated using the attribute vector f of 
the new node V, All duplicated links are removed from 

15 the network by Step 23. This ends the processing of one 

link. Step 24 checks whether there are any more 
unprocessed links by checking if there is more than one 
node remaining in the network. If there are any 
unprocessed links then Steps 20, 21, 22, 23 are 

2 0 repeated, 

A SST is then constructed from the saved links in 
Step 25. This SST represents the entire image in a 
hierarchical organization with its root node being the 
entire image as one segment and its leaf nodes being 

25 block segments. The SST can be partitioned or cut at 

various levels so that the entire image is segmented 
into various numbers of segments. At Step 26 the SST is 
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cut into R sub-trees at its R-1 most costly links. 
Information on the cost assessment of the links can be 
found in most books on algorithms, such as 
"Algorithmics: Theory and Practice," Brassard and 
Bratley, Prentice Hall, 1988. At Step 27, these sub- 
trees are mapped back onto the image space to form R 
segments of the entire image. 

The output from Step 27 is the final segmentation 
of the entire image by using the system 1 and the 
combining method 18 described above. The system 1 can 
be implemented in either a sequential or a parallel 
mode. The number of blocks that can be processed 
concurrently in the parallel mode depends on system 
resources available. 

Referring to FIG. 5 there is illustrated an example 
image 29 comprising thirty-six pixels P that are either 
black or white. A white pixel has an intensity of 1.0 
and a black pixel has an intensity of 0.0. The 
intensity is selected as the attribute to decompose the 
image into segments. Using the segmentation method will 
result in the steps below. 

First, the image 29 is partitioned into four image 
blocks (Block 1 to Block 4) by the image partition 
module 2, each having 3 by 3 pixels. No connectivity 
information of the blocks is passed to the segment 
combination module 5 for such division of the image 29. 
The connectivity information in such a case is implicit 
as the image blocks are of a regular shape and size. 

Then, each image block (Block 1 to Block 4) is 
segmented using the SST technique. FIG. 6 shows the 



process of segmenting Block 1 of FIG. 5. Block 1 is 
mapped into a 4-connectivity network (30) where pixels 
P are labelled from PI to P9 . The weight of each link 
is calculated as the Euclidean distance in intensity 
between two pixels connected by the link. The weight 
value is indicated next to the link in the network 
(30). Networks (31) to (38) in FIG. 6 show steps to find 
a shortest spanning tree of the network (30) . At each 
step, the following operations are conducted: 

1. Find a link with the least weight and save the 
link; 

2 . Merge the two nodes connected by the link into 
one merged node. The size of a merged node in 
pixel is the sum of the sizes of the two nodes 
and the intensity of the merged node is 
calculated using Eq-2 . If a node comprises a 
single pixel, its size is one. 

3 . Recalculate the weights of links that are 
connected to the merged node using Eq-1. 

4. Remove duplicated links. 

These operations are repeated (as shown in networks 31- 
3 8) until all links are processed to leave only one 
node (3 8) in the network. Networks (31) -(38) form the 
hierarchical representation of a SST of the network 
(30) . If two segments are required, the SST should be 
cut at the level that is demonstrated in network (37) , 
Network (37) has two segments; one segment consisting 
of white pixels PI, P2 , P3 , P4, P5 and P7 and another 
consisting of black pixels P6, PB and P9 . 



The block segmentation described above is also 
applicable to the rest of the three image blocks (Block 
2, Block 3 and Block 4), Once all blocks (Block 1 to 
Block 4) have each been segmented into two segments as 
shown in a network 40 of FIG^V, the block segments are 
combined into segments of the entire image 29 using the 
combining method 18 involving the same SST technique 
used in block segmentation. 

The network 40 is mapped from the segments of the 
four image blocks (Block 1 to Block 4) using 4- 
connectivity . In network 40 , nodes Rl, R3 , R5 and R7 
represent the segments formed by the white pixels in 
Blocks Ir 2, 3 and 4 respectively and nodes R2 , R4, R6 
and R8 represent the segments formed by the black 
pixels in Block 1, 2, 3 and 4 respectively. Networks 
(41) -(47) show the steps to construct a SST of the 
network 40. For each step, the same four operations 
described above are applied. If the SST is cut at the 
level shown in network (46) , two segments of the entire 
image can be obtained, one segment is formed by all 
black pixels and another formed by all white pixels. 

Advantageously, the present invention results in 
efficient image segmentation. The efficiency 
measurements for the image segmentation can be 
determined as follows: 



complexity of 0[n^ )x\ — +A; and 




memory requirement of 0{kn)-^B} 
for a sequential mode implementation and 
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complexity of 0(n^j+A; and 
memory requirement of 0{kN) 
for a parallel mode implementation; 
where 

5 N is the size of the entire image; 

p is a constant of a value greater than or equal 
to 2; 

k is a constant of a value greater than 1; 
n is the size of a block; 
10 A accounts for computational overhead associated 

with the combining method; and 

B accounts for the additional memory required for 
storing the block segments. 

Information on determining such efficiency 
15 measurements can be found in most books on algorithms, 

such as "Algorithmics: Theory and Practice," Brassard 
and Bratley, Prentice Hall, 1988. 

Comparing these measurements with those of the 
prior art, it should be noted that: 




0(kn) + B<OikN) 

The larger the size of the original image, N, the 
more significant will be the reduction in execution 
time and memory requirement. 
25 Although the invention has been described with 

reference to the preferred embodiment, it is to be 
understood that the invention is not restricted to the 
embodiment described herein. For example, although the 
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invention was illustrated with reference to a black and 
white image, grey scale or colour images can also be 
segmented. As another example^ block segmentation may 
be performed using methods other than the SST 
technique. The block segments produced can then be 
combined by a method such as the combining method 
described above. 



